{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "db4abe3e",
   "metadata": {},
   "source": [
    "## Preparing Backtest Results for Alphalens"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "37242902",
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install alphalens-reloaded"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "ff0ff1ed",
   "metadata": {},
   "outputs": [],
   "source": [
    "import warnings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "145dda43",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "from alphalens.utils import get_clean_factor_and_forward_returns\n",
    "from IPython.display import Markdown, display"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "f2667469",
   "metadata": {},
   "outputs": [],
   "source": [
    "warnings.filterwarnings(\"ignore\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "83a04bf2",
   "metadata": {},
   "source": [
    "Load the mean reversion data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "fa9ed518",
   "metadata": {},
   "outputs": [],
   "source": [
    "mean_reversion = pd.read_pickle(\"mean_reversion.pickle\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "270b12fb",
   "metadata": {},
   "source": [
    "Construct a dataframe with symbols in the columns and dates in the rows for prices"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "a78dc8e0",
   "metadata": {},
   "outputs": [],
   "source": [
    "prices = pd.concat(\n",
    "    [df.to_frame(d) for d, df in mean_reversion.prices.dropna().items()], axis=1\n",
    ").T"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cab306fc",
   "metadata": {},
   "source": [
    "Convert column names to strings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "ee78226d",
   "metadata": {},
   "outputs": [],
   "source": [
    "prices.columns = [col.symbol for col in prices.columns]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9500c02d",
   "metadata": {},
   "source": [
    "Normalize Timestamp to midnight, preserving tz information"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "281fdc96",
   "metadata": {},
   "outputs": [],
   "source": [
    "prices.index = prices.index.normalize()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a498fbd6",
   "metadata": {},
   "source": [
    "Construct a dataframe with symbols in the columns and factor rank in the rows"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "1a435feb",
   "metadata": {},
   "outputs": [],
   "source": [
    "factor_data = pd.concat(\n",
    "    [df.to_frame(d) for d, df in mean_reversion.factor_data.dropna().items()], axis=1\n",
    ").T"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b52754a1",
   "metadata": {},
   "source": [
    "Convert column names to strings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "9d28c7ab",
   "metadata": {},
   "outputs": [],
   "source": [
    "factor_data.columns = [col.symbol for col in factor_data.columns]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ca68f6d2",
   "metadata": {},
   "source": [
    "Normalize Timestamp to midnight, preserving tz information"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "75cf4459",
   "metadata": {},
   "outputs": [],
   "source": [
    "factor_data.index = factor_data.index.normalize()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "288ffff5",
   "metadata": {},
   "source": [
    "Create a multiindex with date in level 0 and symbol in level 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "17ed8637",
   "metadata": {},
   "outputs": [],
   "source": [
    "factor_data = factor_data.stack()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "73590d57",
   "metadata": {},
   "source": [
    "Rename the multiindexes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "6de29a0e",
   "metadata": {},
   "outputs": [],
   "source": [
    "factor_data.index.names = [\"date\", \"asset\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9c132c04",
   "metadata": {},
   "source": [
    "Display factor data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "2a87ec62",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "date                       asset\n",
       "2016-01-04 00:00:00+00:00  AAL      1156.0\n",
       "                           AAPL     2547.0\n",
       "                           ABBV      438.0\n",
       "                           AET       893.0\n",
       "                           AGN      1371.0\n",
       "                                     ...  \n",
       "2017-12-29 00:00:00+00:00  ISRG     2449.0\n",
       "                           DWDP     1277.0\n",
       "                           ANTM     1510.0\n",
       "                           PCG      2440.0\n",
       "                           CBS       292.0\n",
       "Length: 50275, dtype: float64"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "display(factor_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1466f281",
   "metadata": {},
   "source": [
    "Compile the forward returns, factor rank, and factor quantile using Alphalens"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "0286ccfb",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Dropped 12.5% entries from factor data: 12.5% in forward returns computation and 0.0% in binning phase (set max_loss=0 to see potentially suppressed Exceptions).\n",
      "max_loss is 35.0%, not exceeded: OK!\n"
     ]
    }
   ],
   "source": [
    "alphalens_data = get_clean_factor_and_forward_returns(\n",
    "    factor=factor_data, prices=prices, periods=(5, 10, 21, 63)\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d02ca174",
   "metadata": {},
   "source": [
    "Display Alphalens data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "08679fa9",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>5D</th>\n",
       "      <th>10D</th>\n",
       "      <th>21D</th>\n",
       "      <th>63D</th>\n",
       "      <th>factor</th>\n",
       "      <th>factor_quantile</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>date</th>\n",
       "      <th>asset</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"5\" valign=\"top\">2016-01-04 00:00:00+00:00</th>\n",
       "      <th>AAL</th>\n",
       "      <td>0.004155</td>\n",
       "      <td>-0.050110</td>\n",
       "      <td>-0.037399</td>\n",
       "      <td>0.041066</td>\n",
       "      <td>1156.0</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>AAPL</th>\n",
       "      <td>-0.064737</td>\n",
       "      <td>-0.082487</td>\n",
       "      <td>-0.084670</td>\n",
       "      <td>0.054770</td>\n",
       "      <td>2547.0</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ABBV</th>\n",
       "      <td>-0.064746</td>\n",
       "      <td>-0.045478</td>\n",
       "      <td>-0.055893</td>\n",
       "      <td>0.027773</td>\n",
       "      <td>438.0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>AET</th>\n",
       "      <td>-0.036061</td>\n",
       "      <td>-0.040454</td>\n",
       "      <td>-0.053908</td>\n",
       "      <td>-0.064800</td>\n",
       "      <td>893.0</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>AGN</th>\n",
       "      <td>-0.026344</td>\n",
       "      <td>-0.050997</td>\n",
       "      <td>-0.080561</td>\n",
       "      <td>-0.097310</td>\n",
       "      <td>1371.0</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"5\" valign=\"top\">2017-09-29 00:00:00+00:00</th>\n",
       "      <th>ADP</th>\n",
       "      <td>0.004595</td>\n",
       "      <td>0.004595</td>\n",
       "      <td>0.004595</td>\n",
       "      <td>0.004595</td>\n",
       "      <td>1239.0</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>COL</th>\n",
       "      <td>0.011153</td>\n",
       "      <td>0.024537</td>\n",
       "      <td>0.035920</td>\n",
       "      <td>0.035920</td>\n",
       "      <td>2196.0</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>BBY</th>\n",
       "      <td>0.059996</td>\n",
       "      <td>0.089902</td>\n",
       "      <td>0.089902</td>\n",
       "      <td>0.089902</td>\n",
       "      <td>2486.0</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>EFX</th>\n",
       "      <td>0.025883</td>\n",
       "      <td>0.068132</td>\n",
       "      <td>0.037301</td>\n",
       "      <td>0.027976</td>\n",
       "      <td>2501.0</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>SBAC</th>\n",
       "      <td>0.009967</td>\n",
       "      <td>0.044323</td>\n",
       "      <td>0.044323</td>\n",
       "      <td>0.044323</td>\n",
       "      <td>2390.0</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>43995 rows × 6 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                       5D       10D       21D       63D  \\\n",
       "date                      asset                                           \n",
       "2016-01-04 00:00:00+00:00 AAL    0.004155 -0.050110 -0.037399  0.041066   \n",
       "                          AAPL  -0.064737 -0.082487 -0.084670  0.054770   \n",
       "                          ABBV  -0.064746 -0.045478 -0.055893  0.027773   \n",
       "                          AET   -0.036061 -0.040454 -0.053908 -0.064800   \n",
       "                          AGN   -0.026344 -0.050997 -0.080561 -0.097310   \n",
       "...                                   ...       ...       ...       ...   \n",
       "2017-09-29 00:00:00+00:00 ADP    0.004595  0.004595  0.004595  0.004595   \n",
       "                          COL    0.011153  0.024537  0.035920  0.035920   \n",
       "                          BBY    0.059996  0.089902  0.089902  0.089902   \n",
       "                          EFX    0.025883  0.068132  0.037301  0.027976   \n",
       "                          SBAC   0.009967  0.044323  0.044323  0.044323   \n",
       "\n",
       "                                 factor  factor_quantile  \n",
       "date                      asset                           \n",
       "2016-01-04 00:00:00+00:00 AAL    1156.0                3  \n",
       "                          AAPL   2547.0                5  \n",
       "                          ABBV    438.0                1  \n",
       "                          AET     893.0                2  \n",
       "                          AGN    1371.0                4  \n",
       "...                                 ...              ...  \n",
       "2017-09-29 00:00:00+00:00 ADP    1239.0                3  \n",
       "                          COL    2196.0                4  \n",
       "                          BBY    2486.0                5  \n",
       "                          EFX    2501.0                5  \n",
       "                          SBAC   2390.0                5  \n",
       "\n",
       "[43995 rows x 6 columns]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "display(alphalens_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f832a8ee",
   "metadata": {},
   "source": [
    "**Jason Strimpel** is the founder of <a href='https://pyquantnews.com/'>PyQuant News</a> and co-founder of <a href='https://www.tradeblotter.io/'>Trade Blotter</a>. His career in algorithmic trading spans 20+ years. He previously traded for a Chicago-based hedge fund, was a risk manager at JPMorgan, and managed production risk technology for an energy derivatives trading firm in London. In Singapore, he served as APAC CIO for an agricultural trading firm and built the data science team for a global metals trading firm. Jason holds degrees in Finance and Economics and a Master's in Quantitative Finance from the Illinois Institute of Technology. His career spans America, Europe, and Asia. He shares his expertise through the <a href='https://pyquantnews.com/subscribe-to-the-pyquant-newsletter/'>PyQuant Newsletter</a>, social media, and has taught over 1,000+ algorithmic trading with Python in his popular course **<a href='https://gettingstartedwithpythonforquantfinance.com/'>Getting Started With Python for Quant Finance</a>**. All code is for educational purposes only. Nothing provided here is financial advise. Use at your own risk."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e488ca62-0a71-4286-9e2b-839952de0e18",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "jupytext": {
   "cell_metadata_filter": "-all",
   "main_language": "python",
   "notebook_metadata_filter": "-all"
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
